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ABSTRACT 


Configural  polysampling  offers  an  effective 
alternative  to  both  usual  Monte  Carlo  techniques 
and  small  sample  asymptotics  in  studying  and 
improving  robust  estimates.  We  restrict  our 
attention  here  to  location  estimators  that  are 
location  and  scale  invariant.  The  methods  are 
applicable  to  both  simple  (iid)  and  compound 
(non-iid)  sampling  situations.  Several  possible 
aspects  of  such  studies  include: 

the  determination  of  the  minimum  attainable 
variance  in  each  sampling  situation, 

the  determination  of  the  maximum  attainable 
polyefficiency  over  several  sampling  situa¬ 
tions, 

the  fine-tuning  of  robust  estimates  with  the 
intent  of  increasing  their  polyefficiency  for 
small  n, 

the  identification  of  data  configurations 
where  one  can  a  priori  expect  poor  perfor¬ 
mance  with  certain  estimators. 

Implementation  of  these  ideas  is  being  carried 
forward;  results  will  be  reported  elsewhere. 


1. 


Introduction. 


A  Robustness  Week  was  held  in  Princeton  in  March  1980. 
Considerable  emphasis  was  given  to  the  question  of  what  to 
do  —  that  is,  which  estimators  to  use  --  in  location  prob¬ 
lems  for  samples  of  sizes  n=4  to  3  or  9.  Two  somewhat  dif¬ 
ferent  schools  of  thought  (and  potential  practice)  emerged: 
(1)  the  theoretical  study  of  estimators  by  small  sample 
asymptotic  methods,  and  (2)  experimental  study  via 
conf igyral-polysampl ing  methods,  to  be  discussed  here.  (The 
latter  methods  are  by  no  means  limited  to  such  small  sam¬ 
ples.) 

The  methods  we  discuss  are  applicable  to  both  simple 
and  compound  sampling  situations.  A  simple  situation  con¬ 
sists  of  a  collection  of  n  independent  random  variables 
which  are  identically  distributed: 

Xj-P 

X.  -  F ( — — )  i=l,...,n  . 

For  example,  the  simple  Gaussian  situation  consists  of  a 
random  sample  of  size  n  from  Gau(0,l).  A  sample  of  n  from  a 
mixture  of  Gaussian  distributions  would  also  correspond  to  a 
simple  sampling  situation.  A  compound  situation  consists  of 
a  collection  of  n  independent  random  variables  which  are  not 
identically  distributed: 
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Xj-H 

-  F i  ( — ^ . 

For  example,  a  realization  from  the  one-wild  Gaussian  situa 
tion  consists  of  a  random  sample  of  size  n-1  from  Gau(0,l) 
and  a  single  Gau{0,100)  variate.  The  use  of  compound  situa 
tions  is  important  in  robustness  studies  though  they  have 
been  used  sparingly.  For  a  recent  account  of  some  moment 
calculations  for  compound-situation  order  statistics,  see 
Bruce,  Pregibon  and  Tukey  (1981). 


For  single  situations  (either  simple  or  compound), 
estimator  performance  is  usually  measured  by  relative  effi¬ 
ciency: 

fc  min  attainable  variance  in  situation  Q 

e  CQ  '  a  var( t| situation  Q) 


Conf igural 
numerator 


sampling  methods  are  useful  in  computing  both  th 
and  denominator  of  this  quantity. 


In  terms  of  assessing  the  variability  of  estimators, 
configural  sampling  methods  are  expected  to  provide  order- 
of-magnitude  gains  in  precision,  over  usual  Monte  Carlo 
methods  in  the  location  problem  (e.g.  Hoaglin,  1971). 

Conf igural-polysampl ing  methods  also  supply  an  honest  esti¬ 
mate  of  the  minimum  attainable  variance  (subject  to  the 
usual  invariance  condition)  for  each  sampling  situation 
against  which  one  can  compare  any  desired  estimator. 


When  several  situations  are  considered,  a  useful  meas¬ 
ure  of  performance  is  the  polyefficiency: 
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polyeff(t)  *  min{effA(t),  effg(t),  ...,  eff2(t)} 

Conf igural-polysampl ing  methods  can  be  used  to  determine  the 
maximum  attainable  polyefficiency. 

More  detailed  data  analysis  of  the  configural- 
polysampling  output  for  any  particular  estimator  can  suggest 
improvement  to  that  estimator  as  a  function  of  sample  size 
(e.g.  Should  the  tuning  constant  increase  or  decrease  with 
n?  Should  the  estimate  be  modified  for  small  n?)  and  also 
as  a  function  of  particular  configurations  (For  what  sorts 
of  configurations  should  we  make  what  kinds  of  change?). 

The  opportunities  for  constructing  confidence  intervals 
(either  conditional  or  unconditional)  are  also  great.  See 
(Tukey  1981a)  for  several  possibilities. 

2.  Conf igural-sampl ing  Methods. 

2A.  Conditional  Mean  Squared  Error  (MSE)  Calcul ations. 

Consider  data  Xj,...,xn  from  situation  0,  so  that 
X  -  F-.  (-~ir")  .  Without  loss  of  generality  we  will  take  *i*0  and 

— U 

2 

<r  *1.  Let  y^  <  y2  S.  **•  £  yn  denot®  the  corresponding  order 
statistics.  The  change  of  variables,  for  any  1  <  a^b  <  n: 

r  *  *a 

8  -  yb-*a 

Cj  ■  (yj-r)/s  for  i's  not  a  or  b  . 
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has  Jacobian  s11”2.  The  vector  c  will  be  called  the  sample 
configuration  (this  is  one  of  many  choices  for  a  definition 
of  a  location-and-scale  configuration) .  As  we  are  concerned 
only  with  location-and-scale  invariant  estimators,  we  have 

t  *  t(y)  =  t(r+sc)  *  r  +  s*t(c)  . 

Thus  we  may  think  of  our  estimator  t  as  operating  on  the 
configuration  vector  since,  conditional  on  the  observed  con¬ 
figuration,  t(c)  is  constant.  This  implies  that  the  condi¬ 
tional  MSE{t(y) }  is  given  by 

MSE(t(y)|c}  «  E  {r  +  st(c)|c}2  .  (1) 

I  #  s 

This  is  a  quadratic  function  of  t(c)  and  is  minimized  by 

t  (c)  *  E { rs I c)/E{ s2 I c)  (2) 

o 

leading  to  the  optimally  invariant  estimate  of  p  given  by 

t  (y)  -  r  +  st  (c)  . 
o  o 

This  estimate  has  conditional  MSE  given  by 

MSE{t  (y)|cj  =  E{rs|c)t  (c)+E{r2|c)  .  (3) 

o  o 

Note  that  each  of  the  above  conditional  expectations  are 
naturally  evaluated  as  two  dimensional  integrals  and  can  be 
computed  numerically  with  high  precision  (see  Hoaglin,  1971, 
and  Relies  and  Rogers,  1973,  for  the  case  a*l,  b*n)  . 

2B.  Unconditional  Monte  Carlo  Estimation. 
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For  a  specific  conf iguration ,  the  conditional  mean 
squared  error  of  tQ  is  given  by  (3).  The  estimate  tQ(y)  is 
not  generally  conditionally  unbiased  for  p,  though  the 
unconditional  bias  vanishes  for  symmetric  situations.  Thus, 
the  unconditional  minimum  attainable  variance  at  situation  Q 
is  given  by 

var{tQ(y)}  =  EC(MSE{  tQ  (y)  i  c} }  =  ^cEr  s  ,  c(  t|  (y)  |  c}  .(4) 

This  value  can  be  approximated  by  averaging  MSEft  (y)lc} 
over  a  sample  of  configurations  drawn  from  situation  0, 
leading  to  the  sampling  estimate 

var{t  (y) }  -  ave  MSE{t  I c }  .  (5) 

o  o 

c 

2C.  Assessment  of  other  Estimators  at  each  5 ituation. 

A  location  and  scale  invariant  estimator  t(y)  is  com¬ 
pletely  determined  by  specifying  the  value  t(c)  associated 
with  a  particular  configuration.  The  conditional  mean- 
squared-error  of  the  resulting  estimate  t(y)  =  r  +  st(c)  can 
be  determined  from  (1),  which  simplifies  into  one  quadratic 
function  for  each  situation,  (whose  choice  affects  MSE,  E 

and  t  ) : 
o 

MSE{t(y)|c}  =  MSE{tQ(y)  I  c>  +  E{s2|c}  (t(c)-to(c)  )2  .(<?) 

The  overall  assessment  of  t(y)  at  situation  Q  is 
obtained  by  averaging  (*>)  over  conf igurations: 
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var{t(y)}  =  ave  MSE{t(y)|c}  . 
c 

This  value  is  to  be  compared  to  the  Monte-Carlo  esti¬ 
mate  of  the  minimum  attainable  variance  given  by  (5).  This 
leads  to  the  measure  of  invariant  Q-efficiency 

e f f q { t ( y) }  =  ^ar { tQ (y) }/var{ t(y)  }  , 

where  Q  refers  to  the  particular  situation  at  hand  (for 
example  either  G  =  Gaussian,  S  =  slash,  or  w  *  one-wild). 
This  estimate  of  the  Q-efficiency  differs  from  the  more 
usual  bounds  on  Q-ef f iciencies  offered  by  (a)  an  estimated 
variance  and  the  Cramer-Rao  lower  bound  (which  is  exact  for 
small  samples  only  if  some  estimate  tQ(y)  attains  the  lower 
bound),  or  (b)  the  upper  bound  (relative  Q-efficiency)  based 
on  the  estimated  variance  and  the  estimated  best  of  the 
variances  at  the  situation  among  estimates  so  far  evaluated. 
The  calculations  of  Hoaglin  (1971)  suggest  that  for  small  n 
and  heavy  tails,  the  conservativeness  of  the  Cramer-Rao 
bound  can  be  considerable  -  too  small  by  a  factor  of  2. 

3.  Configural- poly samp ling  Methods. 

So  far,  we  have  treated  a  specific  situation:  (a) 
determining  the  best  one  can  do  and  (b)  assessing  the  per¬ 
formance  of  a  non-optimal  estimator  at  that  situation,  both 
absolutely  and  compared  to  the  best.  Often  one  is,  or 
should  be,  interested  in  the  performance  of  estimators  at 
several  distinct  situations.  Conventional ly,  samples  are 
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drawn  from  each  situation,  and  some  measure  of  performance 
is  computed  at  each.  Such  studies  can  be  made 

1)  more  efficient,  and 

2)  more  informative 

by  employing  configural  polysampling  methods.  As  the  name 
implies,  configurations  rather  than  samples  are  used 
throughout.  In  order  to  use  a  common  collection  of  confi¬ 
gurations,  one  employs  conf igurations  some  of  which  are 
obtained  from  situations  other  than  the  particular  situation 
being  studied.  This  is  possible  only  if  proper  weights  are 
attached  to  each  configuration,  for  use  at  each  situation. 
(These  weights  are  then  functions  of  both  situation  and  con¬ 
figuration.)  In  this  section,  we  first  show  the  relevance  of 
these  methods  in  assessing  single-situation  performance.  We 
then  discuss  their  importance  in  assessing  polysituation 
performance. 

3A.  Single- situation  Performance . 

Consider  a  single  situation  A,  eff  (t)  being  the  meas- 

Pi 

ure  of  performance.  If  all  samples  are  drawn  from  situation 
A,  then  the  methods  of  section  2  provide  the  configural  sam¬ 
pling  estimate  of  var^ft)  given  by 

varft(t)  =  ave  MSEA  { t  (  y)  I  c}  =  J  WSS  { t  (y)  I  c)  dFw  (c) 
c 

where  dF^fc)  is  the  empirical  distribution  function  'vsed 
upon  N  realizations  of  c.  Now  consider  the  implications  of 
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estimating  var^(t)  when  sampling  from  situations  other  than 
A,  say  situation  Q.  If  we  put 

KAIQ  •  ^(d/dFglcI  =  £,(O/f0(c) 

the  method  of  importance  sampling  (e.g.  Hammersley  and 
Handscomb,  1970)  provides  the  expression 

varA(t)  =  varA,Q(t)  •  /c  MSEA( t { y) | c} , QdFg (c) 

which  is  estimated  by  the  weighted  average  over  configura¬ 
tions 


varA(t)  =  varA|^(t)  *  wave  wSSA  { t  ( y)  I  c} 


where  the  weights  implied  in  "wave"  are,  of  course,  WA|0- 

In  general,  we  want  to  consider  polysampling  at  situations  A 

through  Z,  drawing  NA  through  configurations  totalling 

N  =  5N  ,  with  fractions  c(_  =  N  /N.  The  variance  of  t  satis- 
g  g  J  v 

f  ies 


VarA(t)  *  l  «Q  VarA|Q(t) 


and  can,  in  particular,  be  estimated  by 


0arA(t)  *  5  dQ  varA|Q(t) 


Note  that  w. , _  is  not  necessarily  bounded  for  given  A 
a  i  g 

and  Q  —  and  is  likely  to  be  unbounded.  Thus,  in  extreme 
configurations,  where  the  likelihood  ratio  fA/fg  is  very 
large,  the  contribution  to  the  estimated  variance  will  be 
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disproportionately  large.  In  such  cases,  the  variance 
reduction  properties  of  importance  sampling  are  not  main¬ 
tained  and  the  technique  could  be  terribly  inefficient. 

In  order  to  develop  a  more  useful  set  of  weights, 
regard  the  N^+N^+. . .+NZ  configurations  as  a  restricted  (by 
quotas  for  A,B,...,Z)  sample  of  configurations  from 

H(c)  =  c*AFA(c)  +...+  4zFz(c) 

where  the  c(^ ’ s  with  Sd^=l,  are  as  above.  Then,  for  evalua¬ 
tion  at  situation  A  and  sampling  from  H(c),  we  can  put 

WA|H  =  dFA(c)/dH(c)  =  fA(c)/Sc(QfQ(c)  . 
and  write 


varA(t)  =  ^  MSEft{t(y) lc}dFft(c) 

=  Sc  ,vsVt(y)  !c}wA|HdH(c) 


This  expression  can  be  rearranged  to  give 


varAlt>  ’  SVc  "SEAlt(y)lc))wA|(|dF0(c) 


’  StlQ  VSrA|Q(t' 


which  can  be  estimated  by 


Oarft(t)  =  SdQvarA)Q(t) 


where 
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var*j0  *  wave  MSE^{t(y) |c} 

*  c 

with  weight  W^|H.  Thus  we  obtain  an  estimate  of  form  simi¬ 
lar  to  our  previous  one,  though  now  the  weights  are  bounded. 
This  choice  of  weights  will  provide  stable  contributions  to 
the  overall  variance  estimate  even  in  cases  where  extreme 
configurations  are  observed. 

3B.  Polysituation  Performance. 

In  polysampling  studies  one  is  primarily  interested  in 
assessing  the  performance  of  an  estimator  over  all  the  small 
collection  of  situations  considered.  Naturally,  to  judge 
the  usefulness  of  an  estimator,  one  must  know  how  well  the 
best  estimator  performs  across  these  situations.  For  exam¬ 
ple,  for  n*20  and  the  Gaussian,  slash  and  one-wild-Gaussian 
situations,  the  best  known  polyperformance  (as  measured  by 
relative  tri-efficiency)  is  about  93%.  This  does  not 
correspond  to  the  optimal  estimate  for  each  situation,  only 
the  best  estimate  known  to  date. 

Conf igural-polysampl ing  can  help  remove  the  arbitrari¬ 
ness  of  the  "best  known"  attainable  performance,  replacing 
it  by  the  "best"  attainable  performance  (to  the  accuracy 
provided  by  sampling).  When  polyperformance  is  a  function 
of  the  various  individual-situation  sampling  variances, 

Tukey  (1981b)  describes  a  possible  iteration  within  the  fam¬ 
ily  of  Sayeb  estimates.  Essentially  the  method  requires 
finding  the  "best"  estimate  for  each  configuration,  across 
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situations.  This  estimate  is  not  determined  by  the  behavior 
at  this  configuration  alone;  iterative  refinement  on  its 
values  based  on  behavior  at  all  configurations  is  required. 

As  configural  polysampling  determines  overall  estimator 
performance  as  a  combination  of  results  based  on  individual, 
configurations,  the  possibility  of  modifying  existing  esti¬ 
mators  to  obtain  improved  performance  is  an  attractive 
feature.  A  tentative  approach  would  rely  on  some  measure(s) 
which  summarize  configuration  structure  (see,  for  example, 
Tukey  19Slb)  and  then  attempt  to  correlate  these  with  the 
estimator-performance  results.  Of  particular  interest  is 
the  determination  of  configuration  types  where  the  estimator 
performs  poorly.  The  usefulness  of  a  simple-configuration 
summary  characteristic  may  lead  to  new  types  of  adaptive 
estimates,  or  adaptive  modifications  to  existing  estimates. 
See  for  example  Bell's  (1980)  work  in  adaptive  biweight 
scale  estimation. 
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